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In  the  development  of  large-scale  computer  software 
and  in  the  management  of  the  development  process,  it 
is  often  useful  to  model  the  reliability  and  the  cost  of 
development  of  these  software  packages.  There  have 
been  many  papers  that  develop  models  and  show  their 
usefulness  as  management  tools.  The  models  that  use 
Bayesian  methodology  assume  that  a  prior  distribution 
is  given. 

Our  paper  offers  a  methodology  of  assessing  a  prior 
distribution  subjectively.  Two  computer  programs  have 
been  developed  for  this  particular  purpose:  One  as¬ 
sesses  a  subjective  prior  distribution  and  the  other  sug¬ 
gests  a  family  of  probability  functions. 

The  importance  of  consistent  prior  distributions  is 
twofold.  First,  these  distributions  reflect  consistent  ini¬ 
tial  predictions  because  they  are  developed  by  a  struc¬ 
tured  process.  Second,  these  distributions  are  the  start¬ 
ing  point  for  applying  Bayes’  theorem  to  develop  the 
posterior  distribution  by  modifying  the  prior  distribution 
with  actual  data  available  later. 


INTRODUCTION 

During  the  past  decade,  several  probability  distribu¬ 
tions  have  been  used  in  modeling  the  reliability  of 
computer  software.  Among  the  models  proposed  are 
the  exponential  distribution,  the  Rayleigh  distribu¬ 
tion,  and  the  Poisson  distribution  [1-5].  Recently, 
Bayesian  methodology  has  been  proposed  [6-9].  To 
apply  this  method,  a  prior  distribution  is  necessary. 
In  this  paper,  we  offer  a  structured  approach  in 
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subjective  assessment  of  the  probability  distribution. 
Once  this  has  been  done,  the  general  shape  of  the  dis¬ 
tribution  can  be  ascertained.  Then,  the  search  for  the 
mathematical  form  is  greatly  simplified.  For  instance, 
the  probability  distribution  may  be  skewed,  not  exist 
for  negative  values  of  the  random  variable.  This 
would  eliminate  a  class  of  probability  models  like  the 
normal  distribution  and  give  rise  to  a  host  of  others. 
Although  it  is  still  possible  to  select  a  model  from 
many  available  basic  models,  the  selection  process  is 
at  least  based  upon  some  evidence,  namely,  the  opin¬ 
ion  of  the  experts  in  charge  of  developing  the  software 
package. 

Two  computer  programs  were  written: 

1 .  The  first  assesses  a  subjective  prior  distribution  by 
eliciting  answers  to  questions  on  a  cathode  ray 
tube  (crt).  The  answers  to  these  questions  are  used 
to  plot  the  distribution  function  as  well  as  the  den¬ 
sity  function. 

2.  From  the  general  shape  of  these  functions,  the  sec¬ 
ond  suggests  a  family  of  probability  functions.  For 
instance,  an  inverted  gamma  distribution,  a  beta 
distribution,  or  a  lognormal  distribution  might  by 
hypothesized.  Some  of  the  summary  outputs  of  the 
first  program  become  inputs  for  finding  the  param¬ 
eters  of  the  assumed  distributions. 

An  example  of  a  lognormal  distribution  is  used,  but 
other  families  of  distributions  could  have  been  se¬ 
lected  as  well. 

This  paper  does  not  deal  explicitly  with  the  deri¬ 
vation  of  the  posterior  distribution  which  is  found  via 
Bayes'  theorem  in  conjunction  with  incoming  data. 
The  prior  distribution,  however,  is  an  essential  part 
of  finding  the  posterior  distribution.  If  the  prior  dis¬ 
tribution  found  is  integrated  with  test  information  as 
data  become  available,  then  obviously  this  is  more 
complete  information  than  test  information  alone. 
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PREVIOUS  WORK  IN  PROBABILITY 
ASSESSMENT 

More  recently,  decision  theory  has  been  considered 
as  a  general  framework  for  logical  analysis  of  a  deci¬ 
sion  problem  under  uncertainty.  As  such,  consider¬ 
able  attention  has  been  given  to  problem  formulation 
and  methods  for  the  assessment  of  a  prior  distribu¬ 
tion.  For  example,  Schlaifer's  book  [10],  is  largely 
devoted  to  the  formulation  and  prior  analysis  of  de¬ 
cision  problems:  posterior  analysis  is  discussed  only 
in  the  last  part  of  the  text.  Howard  and  his  associates 
(see,  for  example,  [  1 1,12]),  have  emphasized  the  ap¬ 
plication  of  decision  theory  to  complex,  dynamic,  and 
uncertain  decision  problems.  In  dealing  with  these 
problems,  they  have  explicitly  included  the  problem 
formulation  phase  in  the  decision  analysis  cycle. 

Decision  theory,  either  concerned  with  specific 
models  or  general  frameworks,  treats  uncertainty 
through  subjective  probability  and  treats  attitude  to¬ 
ward  risk  through  utility  theory.  Regardless  of 
whether  the  decision  maker  is  concerned  with  prior 
or  posterior  analysis,  the  prior  probability  distribu¬ 
tion,  reflecting  his  quantified  judgments  about  uncer¬ 
tainty,  is  an  indispensable  input  to  the  analysis. 

One  difficulty  associated  with  probability  assess¬ 
ment  is  the  assessor's  inconsistencies  which  often 
occur  in  formulating  a  prior  distribution.  The  question 
of  how  to  discover  and  remove  inconsistencies  is  of 
general  interest  to  decision  analysts.  Another  ques¬ 
tion  of  interest  is  how  to  fit  a  probability  distribution 
using  the  assessed  fractile  in  order  to  make  the  sub¬ 
sequent  analysis  more  tractable.  Both  of  these  ques¬ 
tions  are  addressed  in  this  paper.  The  paper  offers  two 
computer  programs.  The  first  program  allows  a  per¬ 
son  to  interact  with  the  computer  via  a  graphical  de¬ 
vice  [cathode  ray  tube  (crt))  during  the  course  of  es¬ 
tablishing  a  subjective  distribution.  The  second 
program  fits  a  lognormal  distribution  to  the  subjective 
distribution. 

During  recent  years,  subjective  probability  has 
been  studied  by  researchers  in  various  disciplines 
such  as  psychology,  mathematics,  statistics,  engi¬ 
neering,  and  business  administration  (as  evidenced  by 
the  references  at  the  end  of  the  paper).  While  some 
of  these  studies  are  mainly  theoretical  or  philosophi¬ 
cal,  others  are  experimental. 

In  their  text  [  13],  Pratt  et  al.  present  the  method 
of  equally  likely  subintervals.  Subsequently,  Raiffa 
( 14]  illustrates  this  method  in  detail  by  providing  a 
dialogue  between  a  decision  analyst  and  his  client. 
Schlaifer  [10]  advocates  this  method  and  offers  a 
computer  program  for  fitting  a  cumulative  function 
through  assessed  fractiles. 


For  his  experimental  study,  Winkler  ]  15]  devel¬ 
oped  a  questionnaire  using  four  assessment 
techniques: 

1.  cumulative  distribution  function — assessment  of 
fractiles  by  means  of  equally  likely  subintervals  or 
direct  questions  regarding  fractiles, 

2.  hypothetical  future  samples, 

3.  equivalent  prior  sample  information,  and 

4.  probability  density  function. 

He  used  this  questionnaire  to  elicit  prior  distribu¬ 
tions  from  38  selected  subjects  involved  in  his  study. 

The  use  of  penalty  functions,  or  scoring  methods, 
has  been  discussed  by  several  researchers  as  means 
of  encouraging  honest  assessments.  Specifically,  de 
Finetti  [  16]  presents  the  quadratic  scoring  rule.  Sav¬ 
age  [17]  derives  the  general  class  of  strictly  proper 
scoring  rules  by  considering  probabilities  as  special- 
cases  of  rates  of  substitutions.  Winkler  discusses  the 
use  of  scoring  rules  and  other  payoff  schemes  ( 1 8|  and 
reports  his  experimental  results  ( 19]. 

Stael  von  Holstein  and  his  associates  [  12.20]  focus 
on  the  subject  of  eliciting  the  opinions  of  experts  in 
practical  situations  rather  than  laboratory  experi¬ 
ments.  They  discuss  probability  encoding  in  the  con¬ 
text  of  decision  analysis  and  propose  the  use  of  a 
probability  wheel  to  facilitate  the  encoding  process. 

At  the  Reliability  Conference  in  1970,  Lin  and 
Schick  [1]  presented  the  use  of  an  on-line  computer 
system  to  assist  a  person  in  developing  a  prior  distri¬ 
bution  to  represent  his  beliefs.  Although  the  console- 
aided  procedure  is  illustrated  by  a  problem  in  the 
reliability  field,  this  procedure  is  applicable  to  assess¬ 
ment  of  any  prior  distribution.  Since  then,  consider¬ 
able  experience  with  this  procedure  has  been  gained 
from  experiments  involving  students  in  several  statis¬ 
tics  and  decision  theory  classes  at  the  University  of 
Southern  California 

The  present  paper  results  from  the  authors'  con¬ 
tinued  effort  in  making  the  probability  assessment 
more  practical  by  using  modem  electronic  com¬ 
puters.  This  paper  offers  a  newly  designed  computer 
program  which  has  incorporated  the  experience 
gained  from  the  use  of  the  previous  program.  To  sim¬ 
plify  the  assessment  procedure,  the  new  program 

1.  reduces  the  number  of  questions  significantly 
(from  12  to  6), 

2.  is  highly  conversational  and  interactive, 

3.  checks  for  consistency  as  the  user  answers  ques¬ 
tion  by  question, 

4.  uses  graphical  display  rather  than  the  typewriter 
terminal  to  help  the  user  visualize  the  assessment 
process  as  well  as  to  greatly  increase  the  speed  of 
drawing  the  assessed  probability  curves,  and 
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5.  plots  not  only  the  cumulative  function  but  also  the 

density  function. 

Once  a  subjective  distribution  has  been  deter¬ 
mined,  a  second  computer  program  will  fit  a  lognor¬ 
mal  distribution  to  the  subjective  distribution  to  make 
the  subsequent  analysis  of  maintainability  problems 
more  tractable  mathematically. 

METHOD  OF  ASSESSMENT 

Several  methods  have  been  suggested  for  estimating 
prior  distributions  (see,  for  example  [13,15,21]).  Our 
computer  program  makes  use  of  the  method  of 
equally  likely  subintervals,  which  perhaps  is  the  most 
commonly  used  approach.  The  basic  idea  of  this 
method  is  to  ask  the  decision  maker,  at  any  stage,  to 
divide  a  given  interval  into  two  judgmentally  equally 
likely  subintervals. 

To  begin  with,  the  interval  covering  all  possible 
values  of  an  uncertain  quantity  (usually  called  a  ran¬ 
dom  variable)  is  split  into  two  subintervals  and  the 
decision  maker  is  asked  to  choose  which  subinterval 
to  bet  on.  The  dividing  point  is  then  changed  until  a 
point  of  indifference  as  to  betting  on  one  or  the  other 
subinterval  is  reached.  When  this  point  is  reached,  the 
decision  maker  feels  that  it  is  equally  likely  that  the 
actual  value  of  the  uncertain  quantity  will  fall  above 
(to  the  right  of)  or  below  (to  the  left  of)  this  point.  The 
indifference  point,  which  divides  the  entire  interval 
into  two  subintervals  with  equal  probabilities,  is  the 
median.  Next,  the  decision  maker  is  asked  to  specify 
a  point  that  will  further  divide  the  subinterval  to  the 
left  of  the  median  into  two  equally  likely  parts.  This 
new  point  is  the  first  quartile.  Similarly,  the  subinter¬ 
val  to  the  right  of  the  median  may  be  further  divided 
into  two  equally  likely  parts.  The  decision  maker  may 
proceed  in  this  manner  to  divide  any  given  interval 
(generated  previously)  into  two  equally  likely 
subintervals. 

Suppose  we  let  a>  designate  the  Ath  fractile  of  the 
uncertain  quantity  v,  i.e., 

P <  v  <  v*)  -  A.  0  *  A  I. 

Then,  using  the  method  of equally  likely  subintervals, 
the  decision  maker  is  asked  to  respond  to  a  series  of 
questions  that  will  lead  to  a  determination  of  aa  values 
for  such  A  as  0.5,  0.25,  0.75,  etc. 

COMPUTER  PROGRAM 

The  program  stores  a  set  of  questions  for  the  method 
of  equally  likely  subintervals.  These  questions  are 
displayed  successively  on  a  crt:  the  user  responds  to 


the  questions  by  typing  answers  on  a  teletype.  The 
response  to  each  of  the  questions  is  processed  im¬ 
mediately  and  checked  for  logical  consistency. 

Assuming  you  are  the  user  of  the  program,  the  first 
question  calls  for  the  lower  limit  of  the  probability 
distribution  by  asking  you  to: 

Specify  the  largest  value  such  that  you  feel  virtually  cer¬ 
tain  that  the  actual  value  of  the  uncertain  quantity  will 
fall  above  this  value. 

The  second  question,  on  the  other  hand,  calls  for  the 
upper  limit  of  the  distribution  by  asking  you  to: 

Specify  the  smallest  value  such  that  you  feel  virtually 
certain  that  the  actual  value  of  the  uncertain  quantity  will 
fall  below  this  value. 

In  terms  of  the  fractile  notation  described  earlier,  the 
first  question  asks  for  ,v0  and  the  second  question  asks 
for  ,V|.  The  program  will  check  to  see  if  .v0  is  less  than 
A  i  and  if  you  feel  virtually  certain  that  the  actual  value 
of  the  uncertain  quantity  will  lie  in  between  .v«  and  a,. 

The  third  question  asks  you  to  divide  the  interval  de¬ 
fined  by  the  limits  ,v„  and  .v,  into  two  equally  likely 
subintervals.  The  question  says: 

Specify  the  value  such  that  you  feel  it  is  equally  likely 
that  the  actual  value  of  the  uncertain  quantity  will  fall 
above  or  below  this  value. 

The  answer  to  this  question  yields  .vn:>,  which  should 
lie  in  between  v0  and  a,. 

The  fourth  question,  which  calls  for  vu25,  is  as 
follows: 

Suppose  you  were  told  that  actual  value  is  less  than  v„  v 
Specify  the  value  such  that  it  is  equally  likely  that  the 
actual  value  of  the  uncertain  quantity  *  either  above  or 
below  this  value. 

The  program  will  check  to  see  if  this  answer  lies  in 
between  v0  and  vn  r>. 

The  fifth  question,  which  calls  for  a„7:>,  is  the 
following: 

Suppose  you  were  told  the  actual  value  is  greater  than 
v„  v  Specify  the  value  such  that  it  is  equally  likely  that 
the  actual  value  of  the  uncertain  quantity  is  either  above 
or  below  this  value. 

This  answer  is  checked  to  see  if  it  lies  in  between  v„  5 
and  .v,. 

At  this  point,  the  program  further  checks  for  con¬ 
sistency.  Specifically,  it  asks: 

Now.  do  you  feel  it  is  equally  likely  that  the  actual  \alue 
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of  the  uncertain  quantity  will  lie  within  the  interval  be¬ 
tween  ,v0>25  and  r0  7i  or  outside  of  this  interval? 

if  the  check  is  not  met*  the  program  will  direct  you  to 
review  and  revise  each  of  your  previous  answers. 
Otherwise,  the  program  will  proceed  to  ask  you  to 
specify  the  most  likely  value  (the  mode). 

The  assessments  thus  obtained  are  summarized  on 
the  crt.  The  program  then  fits  a  smooth  cumulative 
distribution  function  through  the  assessed  fractiles. 
At  your  request,  it  will  plot  the  cumulative  curve  and 
the  corresponding  density  curve.  If  these  graphs  do 
not  seem  to  reflect  your  judgments  about  the  uncer¬ 
tain  quantity,  you  will  be  guided  by  the  program  to 
revise  your  previous  responses.  Whenever  you  are 
satisfied  with  the  assessed  distribution,  the  mean  and 
the  standard  deviation  are  computed.  In  addition,  you 
may  ask  for  0.005,  0.015,  0.025,  .  .  .  ,  0.995  fractiles 
of  the  distribution. 


COMPUTER  OUTPUT 

To  illustrate  the  computerized  method  of  probability 
assessment  discussed  above,  the  computer  output  of 
an  example  is  presented.  In  this  example,  the  expert 
(italics)  is  asked  to  quantify  judgments  concerning  the 
debugging  hours  for  a  particular  job.  As  we  can  see 
from  this  output,  the  expert  violates  some  of  the  prob¬ 
ability  axioms  and  is  asked  to  revise  his  responses 
several  times. 

THIS  PROGRAM  IS  DESIGNED  TO  ASSIST  YOU  IN 
(A)  QUANTIFYING  YOUR  PROBABILITY  JUDG¬ 
MENTS  CONCERNING  AN  UNCERTAIN  QUAN¬ 
TITY.  (B)  CALCULATING  THE  MEAN  AND  STAN¬ 
DARD  DEVIATION  OF  THE  PROBABILITY 
DISTRIBUTION  OBTAINED  FROM  THIS  QUANTI¬ 
FICATION,  AND  (C)  FITTING  THE  ASSESSED  DIS¬ 
TRIBUTION  TO  A  THEORETICAL  DISTRIBUTION. 
WHAT  IS  THE  UNCERTAIN  QUANTITY  OF  YOUR 
CONCERN  NOW? 

NUMBER  OF  DEBUGGING  HOURS 
PLEASE  RESPOND  TO  THE  FOLLOWING 
QUESTIONS  WITH  YOUR  CAREFUL 
JUDGMENTS: 

(1)  SPECIFY  THE  LARGEST  VALUE  SUCH  THAT 
YOU  FEEL  VIRTUALLY  CERTAIN  THAT  THE 
ACTUAL  VALUE  OF  THE  UNCERTAIN  QUAN¬ 
TITY  WILL  FALL  ABOVE  THIS  VALUE. 

650 

(2)  SPECIFY  THE  SMALLEST  VALUE  SUCH  THAT 
YOU  FEEL  VIRTUALLY  CERTAIN  THAT  THE 


ACTUAL  VALUE  OF  THE  UNCERTAIN  QUAN¬ 
TITY  WILL  FALL  BELOW  THIS  VALUE. 

100 

SORRY,  YOU  HAVE  MISINTERPRETED  THESE 
TWO  QUESTIONS.  QUESTION  (1)  ASKS  FOR  THE 
LOWER  LIMIT  OF  THE  UNCERTAIN  QUANTITY. 
PLEASE  GIVE  YOUR  NEW  ANSWER  TO  (1). 

100 

QUESTION  (2)  ASKS  FOR  THE  UPPER  LIMIT OFTHE 
UNCERTAIN  QUANTITY.  PLEASE  GIVE  YOUR 
NEW  ANSWER  TO  (2). 

650 

YOUR  ANSWERS  TO  THESE  TWO  QUESTIONS 
IMPLY  THAT  YOU  FEEL  VIRTUALLY  CERTAIN 
THAT  THE  ACTUAL  VALUE  OF  THE  UNCERTAIN 
QUANTITY  WILL  LIE  IN  BETWEEN  100  AND650.  DO 
YOU  AGREE? 

NO 

THEN  YOU  MUST  REVISE  YOUR  ANSWERS). 
PLEASE  TYPE  YOUR  NEW  ANSWER  TO  (I). 

50 

PLEASE  TYPE  YOUR  NEW  ANSWER  TO  (2). 

650 

(3)  SPECIFY  THE  VALUE  SUCH  THAT  YOU  FEEL 
IT  IS  EQUALLY  LIKELY  THAT  THE  ACTUAL 
VALUE  OF  THE  UNCERTAIN  QUANTITY 
WILL  FALL  ABOVE  OR  BELOW  THIS  VALUE. 

250 

(4)  SUPPOSE  YOU  WERE  TOLD  THAT  THE  AC¬ 
TUAL  VALUE  IS  LESS  THAN  250.  SPECIFYTHE 
VALUE  SUCH  THAT  IT  IS  EQUALLY  LIKELY 
THAT  THE  ACTUAL  VALUE  OF  THE  UNCER¬ 
TAIN  QUANTITY  IS  EITHER  ABOVE  OR 
BELOW  THIS  VALUE. 

310 

I  BELIEVE  YOU  DIDN'T  ANSWER  THIS  QUESTION 
CAREFULLY.  PLEASE  GIVE  IT  SOME  THOUGHT. 

DO  YOU  NOW  REALIZE  WHAT  YOU  DID  WRONG? 
NO 

YOUR  ANSWER  TO  (4)  SHOULD  HAVE  BEEN  IN  BE¬ 
TWEEN  50  AND  250.  PLEASE  TYPE  YOUR  NEW 
ANSWER. 

130 

(5)  SUPPOSE  YOU  WERE  TOLD  THAT  THE  AC¬ 
TUAL  VALUE  IS  GREATER  THAN  250.  SPEC¬ 
IFY  THE  VALUE  SUCH  THAT  IT  IS  EQUALLY 
LIKELY  THAT  THE  ACTUAL  VALUE  OF  THE 
UNCERTAIN  QUANTITY  IS  EITHER  ABOVE 
OR  BELOW  THIS  VALUE. 

400 
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(6)  NOW,  DO  YOU  FEEL  IT  IS  EQUALLY  LIKELY 
THAT  THE  ACTUAL  VALUE  OF  THE  UNCER¬ 
TAIN  QUANTITY  WILL  LIE  WITHIN  THE  IN¬ 
TERVAL  BETWEEN  130  AND  400  OR  OUTSIDE 
OF  THIS  INTERVAL? 

NO 

SORRY,  YOUR  NO  ANSWER  INDICATES  THAT 
YOUR  ANSWERS  TO  QUESTIONS  (3)  THROUGH  (5) 
MAY  NOT  REFLECT  YOUR  PROBABILITY  JUDG¬ 
MENTS.  SO.  LETS  REVIEW  EACH  OF  THESE 
ANSWERS. 

<3A)  ACCORDING  TO  YOUR  ANSWER  TO  QUES¬ 
TION  (3).  YOU  FEEL  IT  IS  EQUALLY  LIKELY 
THAT  THE  ACTUAL  VALUE  OF  THE  UNCER¬ 
TAIN  QUANTITY  WILL  FALL  ABOVE  OR 
BELOW  250.  DO  YOU  AGREE? 

YES 

(4A)  WE  NOW  LOOK  AT  YOUR  ANSWER  TO  QUES¬ 
TION  (4).  IF  THE  ACTUAL  VALUE  OF  THE  UN¬ 
CERTAIN  QUANTITY  WERE  LESS  THAN  250. 
WOULD  YOU  FEEL  EQUALLY  LIKELY  THAT 
THE  ACTUAL  VALUE  IS  EITHER  ABOVE  OR 
BELOW  130? 

NO 

PLEASE  TYPE  YOUR  NEW  VALUE  FOR  WHICH 
YOU  WOULD  FEEL  EQUALLY  LIKELY  THAT  THE 
ACTUAL  VALUE  OF  THE  UNCERTAIN  QUANTITY 
WILL  FALL  ABOVE  OR  BELOW  THIS  VALUE. 

160 

(5A)  FINALLY,  YOUR  ANSWER  TO  QUESTION  (5) 
IMPLIED  THAT  IF  THE  ACTUAL  VALUE  OF 
THE  UNCERTAIN  QUANTITY  WERE 
GREATER  THAN  250  YOU  WOULD  FEEL 
EQUALLY  LIKELY  THAT  THE  ACTUAL 
VALUE  IS  EITHER  ABOVE  OR  BELOW  400.  DO 
YOU  STILL  AGREE? 

YES 


Table  1 


Cum.  pros. 

Value 

Corresp. 

question 

0.00 

50 

1 

0.25 

160 

4 

0.50 

250 

3 

0.75 

400 

5 

1.00 

650 

2 

Mode 

200 

7 

(8)  GREAT.  YOU  NOW  HAVE  DONE  YOUR  AS¬ 
SESSMENTS  AS  SUMMARIZED. 

DO  YOU  WISH  TO  REVISE  YOUR  ANSWERS? 

NO 

FROM  THESE  ASSESSED  POINTS.  A  SMOOTHED 
CUMULATIVE  DISTRIBUTION  WILL  BE  OB¬ 
TAINED.  WOULD  YOU  LIKE  TO  SEE  THE  GRAPHS 
FOR  THE  DISTRIBUTION  FUNCTION  AND  THE 
DENSITY  FUNCTION? 

YES  [see  Figures  1  and  2] 

(9)  WOULD  YOU  ACCEPT  THESE  GRAPHS  AS 
REASONABLY  ACCURATE  EXPRESSIONS  OF 
YOUR  JUDGMENTS  CONCERNING  THE  UN¬ 
CERTAIN  QUANTITY? 

YES 

(10)  SUMMARY  MEASURES  OF  THIS  DISTRIBU¬ 
TION  ARE 

MEAN  284.9094 

STANDARD  DEVIATION  155.4427 

(11)  DO  YOU  WANT  TO  SEE  THE  VARIOUS 
FRACTILES? 

YES  [see  Table  2] 

(12)  DO  YOU  WANT  TO  FIT  THE  ASSESSED  DIS¬ 
TRIBUTION  TO  A  THEORETICAL  DISTRI¬ 
BUTION? 

NO 


(6A)  NOW,  LET  S  CHECK  THE  CONSISTENCY  OF 
YOUR  REVISED  ANSWERS.  THE  LOGICAL 
CONSISTENCY  REQUIRES  YOU  TO  FEEL 
EQUALLY  LIKELY  THAT  THE  ACTUAL 
VALUE  OF  THE  UNCERTAIN  QUANTITY 
WILL  LIE  WITHIN  THE  INTERVAL  BETWEEN 
160  AND  400 OR  OUTSIDE  OF  THIS  INTERVAL. 
DO  YOU  FEEL  THAT  WAY? 

YES 

(7)  SPECIFY  THE  MOST  LIKELY  VALUE  (THE 
—  MODE),  [see  Table  I]. 

200 


Figure  1.  Distribution  and  density  functions  for  debugging 
hours. 
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Table  2 


0.005 

54.8% 

0.255 

161.7% 

0.015 

59.664 

0.265 

165.364 

0.025 

64.400 

0.275 

168.900 

0.035 

69.104 

0.285 

172.404 

0.045 

73.776 

0.295 

175.876 

0.055 

78.416 

0.305 

179,316 

0.065 

83.024 

0.315 

182.724 

0.075 

87.600 

0.325 

186.100 

0.085 

92.144 

0.335 

189.444 

0.095 

96.656 

0.345 

192.756 

0.105 

101.136 

0.355 

l%.036 

0.115 

105.584 

0.365 

199.284 

0.125 

110.000 

0.375 

200.000 

0.135 

114.368 

0.385 

203.616 

0. 145 

1 18.672 

0.395 

207.264 

0.155 

122.912 

0.405 

210.944 

0.165 

127.088 

0.415 

214.656 

0.175 

131.200 

0.425 

218.400 

0.185 

135.248 

0.435 

222.176 

0. 195 

139.232 

0.445 

226.020 

0.205 

143.152 

0.455 

230.020 

0.215 

147.008 

0.465 

234.180 

0.225 

150.800 

0.475 

238.500 

0.235 

154.528 

0.485 

242.980 

0.245 

158.192 

0.495 

247.620 

0.505 

252.420 

0.755 

403.608 

0.515 

257.380 

0.765 

410.872 

0.525 

262.500 

0.775 

418.200 

0.535 

267.780 

0.785 

425.592 

0.545 

273.220 

0.795 

433.048 

0.555 

278.820 

0.805 

440.568 

0.565 

284.576 

0.815 

448.152 

0.575 

290.400 

0.825 

455.800 

0.585 

2%.  256 

0.835 

463.714 

0.595 

302.144 

0.845 

472.090 

0.605 

308.064 

0.855 

480.930 

0.615 

314.016 

0.865 

490.234 

0.625 

320.000 

0.875 

500.000 

0.635 

326.016 

0.885 

510.080 

0.645 

332.064 

0.895 

520.320 

0.655 

338.144 

0.905 

530.720 

0.665 

344.256 

0.915 

541.280 

0.675 

350.400 

0.925 

552.000 

0.685 

356.576 

0.935 

562.880 

0.695 

362.820 

0.945 

573.920 

0.^05 

369.220 

0.955 

585.120 

0.715 

375.780 

0.965 

5%.  480 

0.725 

382.500 

0.975 

606.000 

0.735 

389.380 

0.985 

619.680 

0.745 

3%. 420 

0.995 

631.500 

(13)  DO  YOU  WISH  TO  QUANTIFY  YOUR  JUDG¬ 
MENTS  CONCERNING  ANY  OTHER  UNCER¬ 
TAIN  QUANTITY? 

NO 

THANK  YOU  FOR  YOUR  COOPERATION.  GOOD¬ 
BYE. 


AN  APPLICATION  FROM  PROGRAM 
VERIFICATION 

From  the  assessment  procedure  given  earlier,  several 
fractile  points,  the  mean,  and  the  standard  deviation 
are  available  in  the  summary  output  of  the  computer 
program.  Any  two  fractile  points,  or  a  fractile  point 
and  the  mean,  or  a  fractile  point  and  the  standard  de¬ 
viation,  etc. ,  can  be  used  to  determine  the  parameters 
of  the  lognormal  distribution.  This  distribution  plays 
an  important  role  in  the  field  of  maintainability.  A  new 


Figure  2.  Cumulative  distribution. 
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program  was  developed  that  allows  some  20  different 
input  combination  pairs  in  the  procedure  for  deter¬ 
mining  the  parameters  of  the  lognormal  distribution. 
The  density  function  of  the  lognormal  distribution  is 
given  by: 


fix) 


Ps/Itt  X 


I  /  In  v  -  a\2 
2\  P  ) 


x  >  0 
/3>0’ 


(1) 


where  a  and  /3  are  the  parameters  of  the  lognormal 
distribution. 


It  is  well  known  that  the  mean  £(.v)  and  the  variance 
V{.\)  are  given  by: 


E(x)  =  fx  =  exp(u  +  [V2)  j3"K 


V(x)  =  tr*  =  p* 


(exp  j32)  -  I 


The  mode  of  this  distribution  is  at 


mode  =  exp(a  -  /J2), 

whereas  the  median  or  50th  percentile  PM)  is  at 

p  ~  t*n 

By  letting  v  =  In  v  -  a//3  in  (1)  and  using  standard 
normal  tables,  the  90th  percentile  was  found  to  be 

Pm  -  exp(  1.282/3  +  a). 

Other  fractile  points  can  be  found  in  a  similar  fashion. 

As  we  have  seen  the  lognormal  distribution  has  two 
parameters  «  and  )3.  Thus  to  fit  a  lognormal  distri¬ 
bution  to  the  subjectively  derived  distribution  we  only 
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have  to  specify  two  values  such  as  P&  and  Pmu  or  the 
mean  and  the  standard  deviation.  For  the  following 
example  the  mode  -  200  and  the  median  =  250  are 
used.  The  program  output  includes  a  distribution 
function  and  a  density  function.  The  latter  is  given  in 
Figure  3. 

LOG  NORMAL  DISTRIBUTION 


DO  YOU  NEED  THE  COMBINATION  PRINTOUT? 
YES=  I.  NO=0  ?0 

WHAT  IS  THE  INPUT  COMBINATION  NUMBER  >13 
MEDIAN  =  7250 
MODE  =  >200 

ALPHA  BETA  MEDIAN  MEAN 
5.5215  0.4724  250.0000  279.5085 

90TH 

STD  DEV  MODE  PCTLE  TIME 
139.7542  200.0000  458.0842 

DO  YOU  WISH  TO  INTEGRATE-NO =0.  YES=  I. 
RETURN  =  2  ?0 

DO  YOU  WISH  TO  PRINT  X  AND  Y-NO=0,  YES  =  1. 
RETURN =2  ?1 

WHAT  IS  XMIN,  XMAX.  DELX 
5100.650.20 


X-VALUES 

Y  VALUES 

X  VALUES 

Y-VALUES 

100 

1  28704E-03 

400 

1 .28704E-03 

120 

2.10476E-03 

420 

1.I00I6E-03 

140 

2.8401  IE-03 

440 

9.37937E-04 

160 

3.378 14E-03 

460 

7.98064E-04 

180 

3.68409E-03 

480 

6.78084E-04 

200 

3.77688E-03 

500 

5.75580E-04 

220 

3.7078E-03 

520 

4.88276E-04 

240 

3.50578E-03 

540 

4. 14090E-04 

260 

3.23704E-03 

560 

3.5U59E-04 

280 

2.93064E-03 

580 

2.97842E-04 

300 

2.61306E-03 

600 

2.52707E-04 

320 

2.30232E-03 

620 

2.14516E-04 

360 

I.74I50E-03 

640 

1 .82209E-04 

380 

1 . 50047 E -03 

DO  YOU  WISH  TO  PLOT  X  AND  Y;  NO=0,  YES=I  ?l  [See 
Figure  3.| 


Now  the  distribution  function  or  density  function 
can  be  visually  compared  with  the  subjectively  de¬ 
rived  prior  distribution  using  the  questionnaire  in¬ 
volving  the  debugging  hours.  If  “reasonable"  agree¬ 
ment  has  been  achieved,  the  mathematical  form  ofthe 
density  has  been  found.  Several  combinations  of 
input  values  might  have  to  be  examined  in  order  to 
achieve  the  “best"  fit.  This  form  is  important  in  order 
to  establish  the  posterior  distribution  using  incoming 
data  and  the  likelihood  function  according  to  Bayes' 
theorem.  On  the  other  hand,  if  “reasonable"  agree¬ 
ment  between  the  two  distribution  functions  has  not 
been  achieved,  a  new  family  of  distributions  may  be 
tried  and/or  the  empirical  distribution  might  be  ques¬ 


Figure  3.  Lognormal  density  function  with  median  =  250 
and  mode  =  200 


tioned.  Ultimately,  agreement  will  be  found  unless 
the  lognormal  distribution  is  not  a  valid  model  de¬ 
scribing  debugging  hours. 
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